Interactive learning map

ABSTRACT

One or more learning map interfaces are displayed. The learning map interfaces include a selection control configured to allow a user to select a subset of assessments from a set of assessments. The subset includes less assessments than the set. Performance information for a student is determined based on the subset of assessments. A learning map is populated with the performance information for the student. The learning map includes a plurality of cells. Each cell corresponds to a learning objective. The plurality of cells is arranged in a manner that reflects a relationship between the learning objectives corresponding to the cells. One or more learning map interfaces are displayed. The learning map interfaces are configured to display the learning map populated with the performance information.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional Application No.61/391,533, filed Oct. 8, 2010, and titled “Interactive Learning Map,”the entire contents of which are hereby incorporated by reference.

FIELD

This document relates to educational software.

BACKGROUND

In an educational setting, such as elementary or secondary school, a setof learning objectives can be identified for a class or other group ofstudents. The identified learning objectives may, for example, be partof one or more academic standards imposed by a nation, a state, adistrict, or some other entity. The learning objectives included in astandard may be predetermined by teachers, administrative staff, schooldistricts, or an expert on teaching the relevant subject matter. Ateacher or other staff member can administer an assessment to determine,for each student in a class, a respective student's abilities withrespect to one or more learning objectives that are covered by theassessment. Evaluation of a student's progress towards acquiring a givenlearning objective may be based on a student's performance on one ormore assessment items pertaining to the corresponding learningobjective.

SUMMARY

In a first aspect, a computing system includes one or more processingdevices and one or more storage devices storing instructions that areexecuted by the one or more processing devices. The instructions, whenexecuted, cause the one or more processing devices to display one ormore learning map interfaces. The learning map interfaces include aselection control configured to allow a user to select a subset ofassessments from a set of assessments. The subset includes lessassessments than the set. The instructions, when executed, cause the oneor more processing devices to determine performance information for astudent based on the subset of assessments. The instructions, whenexecuted, cause the one or more processing devices to populate alearning map with the performance information for the student. Thelearning map includes a plurality of cells. Each cell corresponds to alearning objective. The plurality of cells is arranged in a manner thatreflects a relationship between the learning objectives corresponding tothe cells. The instructions, when executed, cause the one or moreprocessing devices to display one or more learning map interfaces. Thelearning map interfaces are configured to display the learning mappopulated with the performance information.

In a second aspect, a method includes displaying one or more learningmap interfaces. The learning map interfaces include a selection controlconfigured to allow a user to select a subset of assessments from a setof assessments. The subset includes less assessments than the set. Themethod includes determining performance information for a student basedon the subset of assessments. The method includes populating a learningmap with the performance information for the student. The learning mapincludes a plurality of cells. Each cell corresponds to a learningobjective. The plurality of cells are arranged in a manner that reflectsa relationship between the learning objectives corresponding to thecells. The method includes displaying one or more learning mapinterfaces. The learning map interfaces are configured to display thelearning map populated with the performance information.

Implementations of the foregoing aspects can include any, all, or noneof the following features. The selection control can includerepresentations of the assessments in the set of assessments such thatthe user can select the representations to select the subset ofassessments. The selection control can include a timeline control thatallows a user to select a time such that assessments in the set ofassessments administered to the student after the selected time areincluded in the subset of assessments and assessments in the set ofassessments before the selected time are not included in the subset. Thesubset of assessments can include multiple, different types ofassessments. Populating the learning map with performance informationcan include accessing a set of assessment scores corresponding toassessment items administered to the student on the assessments in thesubset of assessments. The assessment items can correspond to aparticular one of the learning objectives. Populating the learning mapwith performance information can include: applying weights to theaccessed assessment scores to generate weighted assessment scores;determining a performance level, from among multiple performance levels,of the student based on the weighted assessment scores; and modifyingthe cell corresponding to the particular learning objective to includean indication of the determined performance level. The weights can varybased on a time at which the assessment items were administered to thestudent. The weights can vary based on a difficulty of the assessmentsitems. The weights can vary based on an importance of the assessment onwhich the assessment items were administered. The weights can vary basedon how well the assessment items differentiate mastery levels.Determining performance information for a student based on the subset ofassessments can include determining a first performance level, fromamong multiple performance levels, of the student based on a first setof assessment items corresponding to a first one of the learningobjectives. The first set of assessment items can be included in a firstset of assessments in the subset of assessments. Determining performanceinformation for a student based on the subset of assessments can includedetermining a second performance level, from among the multipleperformance levels, of the student based on a second set of assessmentitems corresponding to a second one of the learning objectives. Thesecond set of assessment items can be included in a second set ofassessments in the subset of assessments. The first set of assessmentscan include at least one different assessment than the second set ofassessments. Populating a learning map with the performance informationfor the student can include modifying a first one of the cells toinclude a first indication of the first performance level. The firstcell can correspond to the first learning objective. Populating alearning map with the performance information for the student caninclude modifying a second one of the cells to include a secondindication of the second performance level. The second cell cancorrespond to the second learning objective. The subset of assessmentscan include multiple, different types of assessments.

Implementations of any of the described techniques may include a methodor process, an apparatus, a device, a machine, a system, or instructionsstored on a computer-readable storage device. The details of particularimplementations are set forth in the accompanying drawings anddescription below. Other features will be apparent from the followingdescription, including the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a system for generating and displayinginteractive learning maps.

FIGS. 2A-2K and 3A-4B illustrate screenshots of an example of a userinterface employing an interactive learning map.

FIG. 5 is a flow chart showing an example of a process for determiningmastery of a learning objective based on multiple assessments.

FIG. 6 shows examples of generic computer devices that may be used toexecute the actions discussed in this document.

DETAILED DESCRIPTION

FIG. 1 illustrates an example of a system 100 for generating anddisplaying one or more interactive tools for evaluating, planning, andreporting activities undertaken by students and/or teachers. Forexample, a learning map may represent various learning objectives thatare to be obtained by students, and a relationship among those learningobjectives (e.g., a progression of the learning objectives). In general,a learning map may be composed of a plurality of cells, with each cellcorresponding to a learning objective and the plurality of cells may bearranged in a manner that reflects a relationship between the learningobjectives corresponding to the cells. Each learning objective mayencompass, for example, one or more skills (things that students need tobe able to do), one or more concepts (things students need tounderstand), and/or one or more facts (things students need tomemorize).

In some implementations, the learning map can provide feedback in aninteractive interface regarding one or more student's progress towardachieving one or more of the learning objectives. Additionally, thelearning map may provide other feedback to students, parents,administrative staff, school district staff, state or national entities,or other party interested in analyzing student and/or teacherperformance. The provided feedback can include learning objectiverequirements, performance scores and statistics, charts and othergraphics, procedure details, and teacher information, among other data.

Referring to FIG. 1, the system 100 includes a server system 102, ateacher client system 104, an administrative client system 106, one ormore third party providers 108, and an assessment warehouse 110. Ingeneral, various components of the system 100 can store or transmit datapertaining to assessments of students. For example, a teacher canadminister assessments to determine a student's abilities with respectto certain learning objectives, and input the results of the assessmentsinto the teacher client system 104 to transmit the results to otherparts of system 100, such as server system 102 or assessment warehouse110. As another example, a student client system (not shown) can be usedto administer an assessment to a student, capture the results, and sendthe results to other parts of the system 100. As yet another example,assessments may be administered by teachers, but then evaluated orgraded by a third party service that stores the results of theassessments in the assessment warehouse 110 and makes the assessmentresults available, for example, to the server system 102.

The assessments may include diagnostic assessments used to determine abaseline for a student's ability in one or more learning objectives,formative assessments to determine learning gaps throughout a particulartime period, and/or summative assessments to determine a level ofunderstanding achieved at the end of a particular time period.Assessment data may be used by the server system 102 to generate andprovide learning maps that reflect one or more students' performance onassessment items for one or more learning objectives represented by thelearning map.

Each of the client systems 104 and 106, third party providers 108, theassessment warehouse 110, and the server system 102 may be implementedusing, for example, a general-purpose computer capable of responding toand executing instructions in a defined manner, a personal computer, aspecial-purpose computer, a workstation, a server, or a mobile device.Client systems 104 and 106, third party providers 108, assessmentwarehouse 110, and server system 102 may receive instructions from, forexample, a software application, a program, a piece of code, a device, acomputer, a computer system, or a combination thereof, whichindependently or collectively direct operations. The instructions may beembodied permanently or temporarily in any type of machine, component,equipment, or other physical storage medium that is capable of beingused by client systems 104 and 106, third party providers 108,assessment warehouse 110, and server system 102.

In the example shown in FIG. 1, the system 100 includes client systems102 and 104. However, in other implementations, there may be more orfewer client systems. Similarly, a single server system 102 is depictedin FIG. 1; however, in other implementations there may be more than oneserver system 102. For example, each of the systems 104 and 106 mayretrieve data over network 116. The data may include, but is not limitedto assessment information 112 (e.g., 112 a-112 c) or map and toolinformation 114 (e.g., 114 a-c) from third party providers 108 and/orassessment warehouse 110 to generate, display, or report assessment datafor a user. In some implementations, information 112 and 114 canrepresent data for any number of students which can be combined oraggregated by the system 100 according to user preferences and/or systemrequirements.

The assessment information 112 can include multiple, different types ofassessment information. For example, the assessment information 112 caninclude information for interim assessments, state assessments tests,national assessment tests, homework assignments, classwork assignments,class quizzes, diagnostic assessments, formative assessments, summativeassessments, end of chapter tests or quizzes, exit ticket assessments,or other types of assessments. An exit ticket assessment is anassessment which can be used to ensure that students have mastered askill before proceeding to other material.

The server system 102 may include a web server configured to provide aweb-based application that provides interactive tools for evaluating oneor more students' progress toward acquiring one or more of the learningobjectives, developing one or more plans for providing instructionalmaterial to students with respect to the one or more learningobjectives, and/or for developing custom interfaces for such evaluationand planning, as described further below. Assessment of a student'sprogress towards acquiring a given learning objective may be based on astudent's performance on one or more assessment items pertaining to thecorresponding learning objective. The learning objectives may be part ofone or more academic standards imposed by a nation, a state, a district,or other entity. The learning objectives included in a standard may bepredetermined by teachers, administrative staff, school districts, or anexpert on teaching the relevant subject matter.

One example set of standards includes the state of New York's “LearningStandards for Mathematics.” Each standard in the set may include one ormore particular learning objectives and each such learning objective maybe able to be assessed using numerous measurable activities. Forexample, one of the learning objectives in the New York standards(6.N.1) requires a student be able to read and write whole numbers intothe trillions. While one genre of standards is discussed in thisdisclosure, other genres can be deployed throughout system 100,including combinations of standards and third party standards.

The server system 102 may include an application server which hosts oneor more application program interfaces (APIs) to provide academicassessment tools to users. The server 102 may provide services, hardwareresources, and software framework to host services. In someimplementations, the server 102 can implement both server-based andclient-based Web applications. For example, the application server 102can provide server applications (e.g., online services or portals) in adevelopment environment or in an external tool.

The server system 102 can provide access to a number of resources forsystem 100. For example, the server system 102 can store assessment datain a repository 118. The assessment data may include external resultsfrom standardized testing. In some implementations, assessment data mayinclude data captured and stored in learning map structures. Forexample, the repository 118 may store individual student records,standards, and teacher plans.

The teacher client system 104 can provide one or more user interfacesfor teachers to employ. In some implementations, the teacher clientsystem 104 can provide user interfaces for further analysis of studentsor classes. The further analysis may include creating plans to teach,re-teach, or review particular learning objectives in the classroom.

The administrative client system 106 can provide administrative levelaccess to one or more administrators. For example, a principal or schooldistrict manager may access learning maps, teacher plans, and studentdata through the administrative client system 106. In this fashion, anadministrator can oversee teacher and student actions. In someimplementations, the administrator may use the administrative clientsystem 106 for reporting purposes to report out on school or districtperformance. As another example, the teacher client system 104 canprovide a learning map builder user interface. The learning map builderinterface can be used to create a learning map with learning objectivesselected by a teacher or an administrator.

The third party providers 108 may include service and/or contentproviders who provide resources for teachers and other administrativestaff. For example, a third party provider may distribute materialsincluding examples of how to teach different learning objectives. Thematerials may pertain to a specific learning objective, standard,location, or vendor. In some implementations, third party materials canbe provided for a particular cost. In some implementations, third partyproviders can provide information free of charge.

The assessment warehouse 110 may store assessment data for a number ofstudents. The assessment data may include the students' answers toassessment items that assess the student's abilities with respect tovarious learning objectives, whether or not the students answered thoseassessment items correctly, and other information such as standardizedtesting scores, student records, teacher data, learning map structures,and other system information. The assessment warehouse 110 may act as arepository for certain assessments administered by teachers, but scoredby third party services, such as standardized tests.

At a high level, the system 100 can receive assessment data at serversystem 102. The assessment data may be received from any or all of theassessment warehouse 110, the teacher client system 104, oradministrative client system 106 over network 116, for example. Thereceived assessment data can be stored in repository 118 for future use.The stored assessment data can be retrieved from repository 118 topopulate a learning map with performance information related to one ormore students. For example, the server system 102 can use the storeddata to populate and present a learning map and associated tools to theteacher client system 104 and/or the administrative client systems 106.The learning map may be populated by modifying one or more of the cellsin the learning map to include an indication of a performance level,from among multiple performance levels, of one or more students. Inaddition, or as an alternative, the server system may provide learningmap design tools to the teacher client system 104 and/or theadministrative client system 106.

In some implementations, a single assessment, or a number of assessmentsof the same type, may be used to populate the cells with performanceinformation. Alternatively, or additionally, multiple, different typesof assessments may be used to populate more of the cells in the learningmap than could be done, for example, with a single assessment (or groupof assessments of the same type). In other words, multiple, differenttypes of assessments may cover additional learning objectives than wouldbe covered by a single assessment and, therefore, can be used topopulate more cells. Alternatively, or additionally, as described inmore detail below, multiple, different types of assessments can be usedto determine which performance indication with which a given the cell ispopulated.

In some implementations, system 100 is a distributed client/serversystem that spans one or more networks such as network 116. In suchcases, the various components—such as systems 102-110—may communicatevia a virtual private network (VPN), Secure Shell (SSH) tunnel, or othersecure network connection. Accordingly, rather than being delivered aspackaged software, system 100 may represent a hosted solution that mayscale cost-effectively and help drive faster adoption. In this case,portions of the hosted solution may be developed by a first entity,while other components are developed by a second entity. In suchembodiments, data may be communicated or stored in an encrypted formatusing any standard or proprietary encryption algorithm. This encryptedcommunication may be between the user (or the application or client) andthe host or amongst various components of the host. Put simply,communication or other transmission between any modules and/orcomponents may include any encryption, export, translation, or datamassage, compression, and so forth as appropriate. Further, system 100may store some data at a relatively central location (e.g., accessibleover a WAN), while concurrently maintaining local data at the user'ssite for redundancy and to allow processing during downtime.

FIGS. 2A-2K illustrate screenshots of an example of learning map userinterfaces that display an interactive learning map and associatedtools. The user interfaces, and underlying application, may be providedby server system 102 as a web-based application, with the interfacedisplayed, for example, on the teacher client system 104, administrativeclient system 106, or other system with the appropriate permissions.More generally, in various implementations, the user interfaces may beimplemented using a web application running in a web browser or othernetworked application, using a standalone application, or a combinationof both.

The user interfaces can be used for evaluating student performance withrespect to particular learning objectives, and may provide tools forplanning teaching methods, aggregating performance metrics, reviewingstatistics, and providing feedback in a visual and interactive manner.The user interfaces can generally display one or more learning maps withvarying levels of detail. The detail may include text, icons, colors,shapes, symbols, fields, functional buttons or controls, links,indicators, or other preconfigured or user-entered data. The materialdepicted in FIGS. 2A-2K may build upon one or more previous screenshots.However, in other implementations, each screenshot may be generatedand/or navigated to without viewing a previous screenshot.

Referring now to FIG. 2A, a screenshot depicts an example of a userinterface 200 displaying an interactive learning map. The user interface200 includes a learning map graphic 202 and an assessment toolbar 204.The learning map graphic 202 represents a number of learning objectiveswhich are represented by cells. The cells shown in the learning mapgraphic 202 are shaped into hexagons connected to a number of otherhexagons which form one or more honeycomb-like shapes. Each individualhoneycomb may represent an individual learning map structure. In someimplementations, each learning map structure depicts a grouping oflearning objectives for a particular subject matter, such as mathproblem solving, comprehension, or communication, just to name a fewexamples. For example, the learning structures 210, 214, 216, and 218together represent fifth and sixth grade math learning objectivesaccording to the state of New York's math standards. Although the cellsare depicted as hexagons connected into honeycomb-like shapes inscreenshots 2A-2K, other shapes or configurations are possible.

Each cell may represent a learning objective. In some implementations, alearning objective may be a learning objective in an educationalstandard, such as the objective (6.N.1) shown in cell 206. Educationalstandards may include particular learning objectives that are to beacquired by a student or group of students. For example, the objective(6.N.1) corresponds to a learning objective for reading and writingwhole numbers to the trillions. Similarly, a cell 208 represents alearning objective (e.g., 6.N.16) pertaining to adding and subtractingfractions with unlike denominators.

The cells can be arranged in a manner that reflects a relationshipbetween the learning objectives corresponding to the cells with, forinstance, connections between the cells indicating particularrelationships or commonalities between learning objectives. For example,the layout or arrangement of the cells may visually indicate aprogression of learning objectives that reflect a path of typicaldevelopment of students with respect to the learning objectives. Theprogression of learning objectives may begin with learning objectivesthat are considered to be more fundamental and which therefore form thebasis for understanding higher level learning objectives. In someexamples, fundamental learning objectives are generally placed to theleft and bottom of a learning map structure. This can provide a basewith which to build upon. Namely, higher level learning objectives(represented in cells) can be stacked upon the fundamental learningobjectives. Thus, as a user moves from left-bottom to top-right withinthe learning map structure, the user can proceed to learning objectivesthat build upon the previous learning objectives. Thus, for example, thecells form a learning map structure beginning with a fundamentallearning objective (e.g., 6.N.1) in cell 206 and ending with a higherlevel learning objective (e.g., 6.N.20) in cell 212. Each individuallearning map structure 214, 216, and 218 represents other learning mapspertaining to different standards and learning objectives.

In some implementations, the progression of learning objectives fromleft-bottom to top-right may indicate a level of difficulty increasebetween learning objectives. In this example, the difficulty level ofeach corresponding learning objective captured in a cell may increase asthe learning map structure extends upward and/or outward. As such, thelearning objective captured in cell 208 may be more difficult than thelearning objective captured in cell 206.

Each cell can display a number of levels of detail depending on the zoomlevel or user selections. In the user interface 200, each cell depicts afirst level of detail that includes a coded identifier of thecorresponding learning objective in, for example, a particular format(e.g., X.Y.Z). The coded identifier indicates a corresponding learningobjective from a set of learning objectives. In some implementations,the first level of detail shows the cell with only an indication of thecorresponding learning objective (for example, the coded identifier) andan indication of the level of performance of one or more students on oneor more assessments of the corresponding learning objective (forinstance, using the color coding described below). In someimplementations, a user can zoom using a zoom toolbar 220 to view moreor less detail in, on, or around each cell. Zooming in and out will bediscussed in detail below.

Each cell can be color-coded or pattern-coded depending on performanceresults attained by a student, a class, a district, a state, or othergrouping of performers on assessment items for the correspondinglearning objective. The color-coding may indicate a level of performanceof one or more students on one or more assessment items for acorresponding learning objective. For example, at the individual studentlevel, green may be associated with mastery of the learning objective,yellow with “at risk” for not mastering the learning objective, and redwith a failure to master the learning objective. In one example, if astudent has obtained a performance score of greater than 85 percent,then the student may be considered as having mastered the learningobjective. For example, if there are a certain number of pointsassociated with assessment items that assess a given learning objective,and a student obtains 85 percent of the points by answering certainassessment items correctly, then the student may be classified asmastering the learning objective. If a particular student has obtained aperformance score between 70 percent and 85 percent, the student may beat risk for not fully learning the objective. Lastly, if a studentobtains a performance score of less than 70 percent, the student may beconsidered as failing to master the learning objective.

Similarly, for a given group of students, green may indicate that thegroup has met a mastery target; yellow may indicate that the group isapproaching a mastery target; and red may indicate that the group isfailing to approach the mastery target. Students may be grouped based onclass, teacher, school, district, state, demographics, such as sex orrace/ethnicity, program participation such as ELL or IndividualizedEducation Program (IEP), intervention groups, or ad hoc groups createdby a user (e.g., the basketball team).

A given group of students' performance level (for example, at masterytarget, approaching mastery target, or failing mastery target) may bebased on the percentage of students in the group who are deemed to havemastered each learning objective. For example, mastery of the learningobjective at the student level may be defined as answering correctly atleast two out of three questions within a particular standard. At theclass level (or other grouping of students), level mastery may bedefined in terms of the percentage of students who met this threshold.For example, a class might be deemed to have met a mastery target for anobjective if 85% of the students in the class were classified as havingmastered the objective. An alternative approach may base a given groupof students' performance on the average points per student in the group.For instance, if a set of assessment items for the learning objectivehas an associated number of points that can be earned by answering theassessment items correctly, then average number of points earned perstudent can be determined, and the class's collective performance may bedetermined based on the percentage of total points to which the averagecorresponds (for example, if the average number of points is above 85percent of the total points, the class may be classified as learningobjective having met a performance target). In some implementations,performance can be described as pass or fail. Other calculations,percentage requirements, statistics, or rules can be used to makeperformance determinations by learning objective, which can then bedepicted visually on the learning map.

In the example shown, the learning maps structures 210, 214, 216, and218 are being displayed for a group of students (as opposed to a singlestudent) and the colors of the cells correspond to the performance levelof the group as a whole on the corresponding learning objectives. Forexample, the learning objective 6.N.14 represented by cell 222 iscolored green to illustrate the class has reached the mastery target ofthis learning objective, the learning objective 6.N.1 represented bycell 206 is colored yellow to illustrate of the class is approaching themastery target, the learning objective 6.N.16 represented by cell 208 iscolored red to indicate the failure of the class to reach the masterytarget for the learning objective. If there is no correspondingassessment data, or a performance level is not otherwise available forsome reason for a given learning objective, then the corresponding cellmay be colored gray or some other color.

The color-coding can be used to analyze performance results to quicklydetermine where issues may require further action with respect toparticular students and/or curriculum. For example, the color-coding canprovide visual feedback to a teacher who can determine whether he or sheshould plan to re-teach a learning objective to a small group ofstudents, or an entire class.

Other navigational and configuring tools can be included in userinterface 200. For example, a panning control 224 can be used to panaround a number of learning map structures. In a similar fashion, theuser can pan around the learning map structures by selecting anddragging inside the learning map window. Other tactile or manual inputmechanisms can be used to accomplish the same panning feature. Forexample, the learning map application may be presented on a touchscreenwhich can enable the user to use his or her touch input to modify aparticular view.

The user interface 200 also includes a legend 225 (not fully shown) toensure the user can properly read each learning map structure. The userinterface 200 additionally includes the assessment toolbar 204 to enablethe user to select the data for a particular learning map view. Forexample, in user interface 200, a user selected a specific school“Forest Green Middle School” from a dropdown control 226. In addition,the user selected grade “6,” class “549,” subject “Math,” assessment“IA,” period “3,” and Student “All Students” from respective dropdowncontrols 228, 230, 232, 234, 236, and 238. In other examples, the usermay select a different grade from dropdown control 228, a differentclass from dropdown control 230, or a different period from dropdowncontrol 236. Other menu selections can be offered and selected.

The user may additionally select a different assessment from dropdowncontrol 234. In this example, the user selected an assessment labeled“IA#3” which indicates an interim assessment for the third period of aschool year for one particular course. In some implementations, theassessment may not be as specific. For example, assessments may beidentified by a name or other descriptor to remove the assumption of howa district structures assessments. Examples of assessments can include,but are limited to, benchmark or interim assessments which may be givenat fixed intervals to all students in a district who are taking the samecourse (e.g., 5th grade math or Algebra 1). Other assessments may bead-hoc in nature and include teacher-created assessments, third partyassessments, administrator-created assessments, among others.

In some implementations, the user may select a particular student orsubset of students rather than “all students” from dropdown control 238.Upon selecting some or all information in the assessment toolbar 204,the user can select a done button 240 to commit the selections to atitle banner 242. Upon committing the selections, the system 100 canupdate the banner 242 and update the display of the learning maps 210,214, 216, and 218 to reflect the data corresponding to the newselection. In addition, a user can choose to minimize or shrink theassessment toolbar 204 by selecting a minimize control 244.

FIG. 2B illustrates a screenshot showing the user interface 200 zoomedinto another level of detail. In this example, the user has selected thezoom control 220 (FIG. 2A) one or more times to zoom in on the learningmap structures. The zoom action may apply to a single learning mapstructure, multiple learning map structures, or individual cells. Thezoom action in this example zoomed all learning map structures to revealfurther details in each cell. The zooming action can, for example,display the learning map 210 at a second level of detail, which shows amodification of the content of a cell. In particular, the zoom actionprovides additional indications of information related to the learningobjective in addition to the (a) coded identifier indicating thelearning objective (e.g., 6.N.16) and (b) the indication of the level ofperformance (e.g., red cell). For example, the learning objective(6.N.16) is now shown as three separate rows of data within the cell208. A first row 246 includes the original coded identifier of thelearning objective (e.g., 6.N.16). A second row 248 includes an icon, orother symbolic indication, or descriptive text of the correspondinglearning objective to potentially convey the corresponding learningobjective to the teacher more easily than the coded identifier. Namely,the second row 248 depicts a math problem involving two fractions withuncommon denominators (e.g., ¼+⅝), illustrating the addition offractions of uncommon denominators, which corresponds to the learningobjective 6.N.16. A third row 250 depicts a plan for the learningobjective in the future. In particular, the third row 250 includes ateacher's plan to “Re-teach” the material in the learning objective(6.N.16) to the whole class. The teacher's plan typically includesinstructions regarding a particular learning objective. The teacher'splan may be automatically dictated by student scores or selected by ateacher or administrator. The plan here may be to re-teach the materialbecause the red color of the cell indicates that a desired number ofstudents did not attain mastery of the learning objective (6.N.16).Planning functionality will be discussed in detail below.

As another example, the cell 222 holding the learning objective (6.N.14)includes a middle row 252 with a graphical representation of thecorresponding learning objective. Here, the graphic is an integer numberline indicating that a student should be able to locate rational numberson a number line. Other shapes, icons, text, or symbols can be combinedon cells.

In some implementations, the bottom rows of the cells can beindividually color-coded to indicate something visually to a teacher.For example, the bottom rows may be color coded in a way thatcorresponds to the particular teaching plan associated with the learningobjective.

FIG. 2C illustrates a screenshot showing the user interface 200 zoomedinto another level of detail. At this level of detail, the icons mayinclude features that indicate whether an entire group of students is atrisk of not knowing a learning objective, and what the relative split isamong students in the group. In particular, the features may allow ateacher to quickly determine the class breakdown in terms of how manystudents have mastered the learning objective.

As shown, the cell 208 now includes the same three rows 246, 248, and250 with additional data and/or functionality. For example, row 246includes the original coded identifier (e.g., 6.N.16) with an added notesymbol 256. The note symbol 256 may indicate that further details ornotes associated with the cell 208 are currently available for viewing.To view the additional details, a user may select the note symbol 256.Upon selecting the note symbol 256, additional details can be displayedto the user in a pop-up dialog, an overlay, a note, a comment, a webpage, or other presenting mechanism. Examples of notes and additionaldetail presentations will be discussed below in FIGS. 2F-2K.

The cell 208 also includes an update to row 248. The update shows therow 248 now semi-divided into two sections with the original mathproblem symbol on the left hand side of the divider and a pie chart 257on the right hand side of the divider. The pie chart 257 represents thespread of performance results amongst students testing on the learningobjectives (6.N.16) represented by the cell 208. Thus, the overall color(e.g., red) of the row 248 of cell 208 may illustrate the performance ofthe entire group, while the pie chart 257 concurrently indicates aproportion of students (i) at risk (ii) at mastery, or (iii) atnon-mastery. That is, the color of the row 248 may indicate a level ofperformance for the entire group, as described above, while acorresponding pie chart or other graphic may indicate a proportion ofmultiple students at one or more levels of performance.

In the example shown, the pie chart 257 depicts three color-codedportions: (a) a proportion 258 of students who have mastered thecorresponding learning objective (e.g., color-coded green), (b) aproportion 259 of students who are at risk for not knowing thecorresponding learning objective (e.g., color-coded yellow), and (c) aproportion 260 of students who have yet to master the correspondinglearning objective (e.g., color-coded red). Furthermore, the pie chart257 indicates that a large percentage of the class have yet to masterthe learning objective represented by cell 208 (i.e., the largest piepiece is red), and as such, row 248 of the cell 208 is color-coded redto indicate this statistic. Thus, the row 248 provides the advantage ofpresenting both how well the class is doing together with a percentagepie chart showing how many students are doing well versus not well onthe learning objective. The concurrent display of an indication of thegroup-oriented performance level of the group of students and anindication of the proportion of the students in the group at themultiple student-oriented performance levels can eliminate amisunderstanding of whether the issue is a large portion of a class orwith just a select few students. This particular visualization mayprovide an easy mechanism with which to decipher class versus individualperformance at a quick glance.

The row 248 additionally includes a power objective icon 261. The powerobjective icon 261 may indicate that the learning objective (6.N.16)represented by cell 208 is a power objective. Power objectives representa subset of learning objectives at each grade level that are the mostimportant for students to master. Power objectives may represent aminimum foundation without which students may have great difficultymastering later objectives. In some examples, power objectives representa “safety net” of objectives each student should learn prior to leavinga particular grade level. The power objectives can be used to ensurethat students who acquire this “safety net” of understanding, knowledge,and skills exit one grade better prepared for the next grade. The powerobjective icon 261 can ensure a teacher understands at a glance whichobjectives are required to move a student or class to the next level.The teacher can then, for example, use this information for planningpurposes.

The row 248 also includes a cell flip control 262. The cell flip control262 can be employed to flip or “turn over” a cell. In one example, theflip of a cell may be graphically shown as a 180 degree turn around avertical axis. In the depicted example, a user can select the cell flipcontrol 262 to reveal more information about a particular class. In someimplementations, selecting the flip control provides information (e.g.,questions and answers) that caused particular cells to be color-coded ina specific manner.

The cell 208 includes the row 250. The row 250 includes a teaching planicon 263 which indicates a selected teaching plan for the learningobjective, which, in this case, is “class re-teach” (described below).As another example, the teaching plan icon 264 shown in cell 265indicates a “review” teaching plan (as described below).

The row 250 also includes an average indicator 266 which indicates anaverage number of points earned per student for a learning objective,and the total number of points. The average indicator 266 shows anaverage score of 3.15 points out of 5 points. The average in thisexample is average over 5 points because the assessment items for thelearning objective (6.N.16) represented by cell 208 had a correspondingtotal of 5 points. In some cases, the total number of questions mayequal the total number of points achievable for a particular learningobjective, while in other cases there may be more or less questions thanthe total number of points (for instance, if one question is worth morethan a point or worth only a fraction of a point).

FIG. 2D illustrates a screenshot showing the user interface 200 where auser has changed a selection in the student dropdown control 238. Asshown, the user selected one particular student “Jason Barnett” insteadof the “All Students” selection described above. The cells shown in FIG.2D pertain to the student “Jason Barnett.” The cell 208 includes rows246, 248, and 250 as shown in FIG. 2C. The information depicted issimilar to the previous screens (learning objective, performance level,teaching plan), with the exception of row 250, and the interface canprovide the same zooming functionality to provide different levels ofdetail when a single student is displayed instead of a group. The row250 includes text 268 stating “4 items.” The items pertain to fourassessment items (e.g., questions) used to assess a student against thelearning objective. Similar functionality exists for selecting the notesymbol 256 and selecting the cell flip control 262.

At some point, a user can select the cell flip control 262 to revealinformation about the performance of the student “Jason Barnett”corresponding to the learning objective associated with the cell 208.

FIG. 2E illustrates a screenshot showing the cell 208 after activatingicon flip control 262. The cell 208 here shows four numbered items 269(#16), 270 (#17), 271 (#18), and 272 (#19). Each numbered item 269-272may correspond to an assessment item (e.g., question) that was used totest Jason Barnett's ability in relation to the learning objective(6.N.16).

A number of symbols are shown near each numbered item 269-272. Thesymbols may depict whether the student obtained the points associatedwith the assessment items 269-272, for example, by answering theassessment item correctly. For example, a shaded symbol, as shown nearnumbered item 269 and 270, may indicate that Jason Barnett received thepoints associated with the two items, for instance, by answering themcorrectly. Similarly, a non-shaded symbol, as shown near numbered item271, may indicate that Jason Barnett inaccurately answered the questioncorresponding to numbered item 271 and therefore did not obtain thecorresponding points. In some implementations, multiple points may beassociated with a single numbered item. For example, numbered item 272includes two shaded symbols indicating that two points may havecorresponded to numbered item 272. Here, Jason Barnett accuratelyanswered the assessment item in such a way that he received both points.A summary of Jason Barnett's assessment is shown by text 273. The text273 depicts a score of 4 out of 5 points which corresponds to 4 shadedsymbols and 1 non-shaded symbol in the cell 208.

FIGS. 2F-2H illustrate example screenshots showing an overlay 274related to cell 208. The overlay 274 can be navigated to by selectingthe cell 208 shown in FIG. 2D, for instance, by clicking a mouse overthe icon 208. The overlay 274 illustrates details about the student“Jason Barnett,” as indicated by tab 275. The overlay 274, or similaroverlay, may similarly be provided when the icon 208 in FIG. 2C(relevant to all students) is selected. Such an overlay may provide thesame, or similar, functionality with respect to the group of students asoverlay 274.

Referring now to FIG. 2F, the overlay 274 includes a graphic 276 of thecell 208. The graphic 276 represents the state of the cell 208 before auser selected the cell. The overlay 274 includes an interactive detailarea 278 that can be accessed by teachers or other staff. The detailarea 278 can include a variety of details. For example, the detail area278 includes the text of the standard description (e.g., “6.N.16—Add andsubtract fractions with unlike denominators”). Other descriptive detailscan be included in reference to the standard or associated learningobjectives. For example, a period notation may indicate the period forwhich data is being displayed.

The detail area 278 may also include one or more resource links 279. Theresource links 279 may be links to teaching resources related to thecorresponding learning objective, such as teaching tools, teachingmaterials, ideas, curriculum content, lesson plans, videos, or othercontent specific to the learning objective corresponding to the selectedcell (in this example, for standard 6.N.16). In general, the materialsmay assist teachers in developing lesson plans for the learningobjective, as well as addressing student issues with actually learningthe objective. In some implementations, the resource links 279 may beplaced directly on a cell, rather than in an overlay. The resource links279 are configured to cause, when selected, a display of the teachingresources related to the learning objective. For instance, in oneimplementation, when the user (for example, a teacher) selects theresource link 279, a web page containing the materials is retrieved anddisplayed.

The resource links 279, and underlying resources, may be intelligentlyfiltered or ordered. For example, the various resources may be reviewedand rated by other teachers (or other users) and those resources above acertain rating may be selected to include on the overlay or cell, or thevarious resources may be included on the overlay or cell in an orderedlist, ordered based on the ratings. Other factors may be used to selectthe resources to link to, or to order the resource links. For example,if the teacher is part of a social network, for instance, implemented bysystem 100, then the fact that a member of the viewing teacher's socialnetwork has used or rated a particular resource may be used whenfiltering or ordering the resource link.

The resources and associated links may be filtered based on geographyand/or the instructional materials used by the school and/or schooldistrict. For example, if the teacher is located in a particular state,then the linked resources may be those relevant to the standards of thatstate, or relevant to particular teaching methods approved in thatstate. Similarly, if a particular set of instructional materials areemployed by a school or district to teach the subject matter reflectedby the learning map, then the resources may be those that supplementthose instructional materials.

The resources may be provided by third party providers 108. In general,system 100 may provide a platform for the third party providers todeliver the resources to the relevant recipients. For example, thesystem 100 may provide a mechanism for the third party to designate thethird party's resources, and indicate what types of learning objectivesto which the resources are relevant, as well as other information aboutthe relevance of the resources such as particular geography orinstructional materials. The system 100 then may select the resources asappropriate based on this information, as described above. Furthermore,the third party providers 108 may want to charge the teacher, school, orschool district for access to some or all of the resources provided bythe third party. The system 100 may provide a mechanism that allows theteacher, school, or district to purchase access to the third party'sresources.

In such a scenario, certain free resources provided by a given thirdparty may be linked to by resource links so that a teacher oradministrator can access the resource while looking at a learning mapand evaluate the usefulness of the resources provided by the thirdparty. The user interface (or another interface) may provide the teacheror administrator with a mechanism for then purchasing full access to thethird party's library of resources, which will then be linked to by thelearning map.

Other filtering tasks and/or ordering of links can be performed.Depending upon how much metadata is available for particular resources,and how much student data is available, there are various tasks that canbe performed. For example, the system may inspect the student data tofind students who are struggling with a particular learning objective,and see if they are ELL students (English Language Learners, i.e.,students whose native language is not English). If the system determinesthe students are indeed ELLs, then the system could suggest resourcesthat have been written with that particular population in mind, orotherwise validated as appropriate for the needs of that population. Insome implementations, the system may inspect student data to determine alearning disability, for example. Similarly, depending on the number ofstudents who need remediation, the system could suggest resources thatare appropriate for large group (e.g., whole class), small group, orone-on-one instructional settings, whichever is most pertinent to thesituation. These general concepts could be applied to bothteacher-delivered instructional resources (e.g., lesson plans,activities, etc.) and computer-delivered instructional resources (e.g.,computer-tutorials). For example, if a particular student's nativelanguages are available to the system, then the system could locate andrecommend computer-based instructional activities that are in thatlanguage. If there are designated funds that are allocated to particularstudents (e.g., Title I or IDEA funds), then the system could take thatinto account in recommending effective ways to spend that money (e.g.,specialized computer-based instruction for students with particularlearning disabilities). In some implementations, a district could put inplace a contract in which the use of various vendors' instructionalresources is pre-authorized, but where the vendor is only paid whenteachers actually select those resources to download and use, and wherethe amount of the payment is dependent upon the amount of use.

The detail area 278 includes a teaching plan selector 280 that may beconfigured to receive an indication of a teaching plan for the learningobjective corresponding to the cell. The teaching plan then may bestored in association with the cell.

In general, the plan selector 280 provides a tool for teachers to selecta teaching plan for one or more individual learning objectives, normallybased on a group's or individual student's performance with respect tothe learning objective. The plan selector may include predefined typesof teaching plans that can be selected by the teacher. In someimplementations, the lesson plan may include corresponding visualrepresentation (not shown). The plan selector 280 is described furtherwith respect to FIG. 2G.

The detail area 278 includes a note area 281. The note area 281 can beused by a teacher to add one or more notes regarding the selectedteaching plan and/or previous teaching activities. The note area 281,for example, may allow a teacher to capture information that providesmore detail about the predefined teaching plan selected by the teacher.Notes can be edited or appended to by selecting an edit control 281. Insome implementations, the note area 281 can be used to keep notes on aparticular student or class. The notes can be accessed by teachers orother users from another view outside of the overlay 274. For example,if a note has been entered in a particular overlay, the correspondingcell will include a note symbol, such as note symbol 256 (FIG. 2D).

The overlay 274 includes a scroll area 282. The scroll area 282 providesa mechanism to scroll through and view other cells from a relatedlearning map. For example, the cell 222 introduced in FIG. 2A is shownin scroll area 282. The cell 222 can be selected by a user. Theselection can cause the cell 222 to be displayed in place of the cell276. Furthermore, upon selection of cell 222, details in the detail area278 may be updated to reflect details about the cell 222. Other cellscan be selected. In addition, the scroll area 282 can be scrolled leftusing a control 283 or right using control 284 to present moreselectable cell options.

The overlay 274 includes a period selector 285 and an order selector286. The period selector 285 provides a mechanism for selecting one ormore assessments. The period selector 285 may be based on the assumptionof a fixed sequence of interim or benchmark assessments, such that“assessment 3” was a meaningful term. In some implementations, thesystem may ask users to choose an assessment from a list, rather thanassuming that a single-digit integer can be used to meaningfullyidentify a particular assessment.

The order selector 286 may determine the order in which the system woulddisplay the learning objectives, if a user were browsing through them.Here, for example, the user has selected (6.N.16). However, the user maywish to review objectives one by one in a linear fashion. In thisexample, the user can review objectives by selecting options such asbefore and after in the order selector 286. The order selector 286 maydefine what “before” and “after” implies according to the objectives. Inthis example, if a user selects a “Standard” option, then the objectivesappear in their native order (e.g., the predecessors would be 6.N.14 and6.N.15, while the successors would be 6.N.17 and 6.N.18). If the userselects a “% points” option, then the objectives would be sorted basedon how a class performed (e.g., the predecessors would be the twoobjectives on which the class scored the next worst after the selectedobjective, while the successors would be the two objectives on which theclass scored the next best after the selected objective). Other sortorders are also possible. For example, the objects may be ordered basedon how the objectives are laid out on the learning map.

Referring now to FIG. 2G, the overlay 274 is shown with the planselector 280 showing options for a teacher to select a teaching plan.The plan selector can provide selectable options for predefined teachingplans including, for example, “Future”, “New Teach”, “Re-teach-WholeClass”, “Re-teach-Small Group”, “Review”, and “Unassigned.” The “Future”plan may be selected to set a plan to teach a particular standard in thefuture. The “Future” plan can be selected based on what standards shouldbe taught based on preset scope and sequence calendar in one or moreperiods following the next period. A “scope and sequence” may be adivision of a set of learning objectives into subsets each of whichcorresponds to a benchmark assessment. For example, if there are 5benchmark assessments over the course of a school year, then thelearning objectives that are supposed to be covered or introduced duringthe period before benchmark #1 constitute the scope for period 1, thosethat are supposed to be covered or introduced during the period betweenbenchmark #1 and benchmark #2 constitute the scope for period 2, and soforth. The “sequence” generally refers to the ordering within theobjectives for any given period. This may represent a logical way ofthinking about and structuring the objectives that need to be coveredover the course of a given course, although some school systems may notfollow this sort of approach (although some do). Accordingly, in someimplementations a more flexible approach may be employed.

The “New Teach” plan may be selected to designate what standards shouldbe taught in the next period based on a preset scope and sequencecalendar. The “Re-teach-Whole Class” plan can be selected if a teacherwishes to re-administer instruction based on some portion of the classnot reaching a certain level of achievement on a learning objective. Thecriteria for when to select a re-teaching plan may be set or preset by aschool district, school, or teacher. The “Re-teach-Small Group” plan canbe selected if a teacher wishes to re-administer instruction to a subsetof students that fall below a pre-determined percentage of the class,for example. The “Review” plan can be selected if a teacher wishes tore-teach a topic even if all the students have successfully mastered aparticular learning objective. For instance, the teacher may select the“Review” plan if a particular learning objective is a “power objective.”The re-teaching process can ensure the “power standard” learningobjective is solid before moving on to other topics and/or learningobjectives. The “Unassigned” plan may be assigned by default untilanother entity modifies the plan.

In the example shown in FIG. 2G, the user selected a “Re-teach-SmallGroup” plan, as indicated by the highlighted selection. The user canselect a plan for the objective (6.N.16) and choose to exit the overlay274, enter notes pertaining to standard (6.N.16), or select another oneor more cells from scroll area 282 for configuration. In someimplementations, an initial plan may be chosen for a given cell based onthe assessment data, but a user can then select another plan, forexample, after reviewing the specific assessment items and/or thestudent answers to those items.

Referring now to FIG. 2H, the overlay 274 is shown with the“Re-teach-Small Group” plan selected. The user may choose to enter anote pertaining to the student's performance, the class's performance, acombination of both, and/or notes about how to implement the plan. Theuser can select the edit control 281 (FIG. 2G) to enter a note 287. Thenote 287 indicates that “Some students did fine with simple problems (½.. . ⅓), both multiple choice and constructed response but were thrown bysmaller fractions ( 1/80). Need to reinforce the concept with group.”The note 287 explains what will be re-taught and why. The user canchoose to cancel the note 287 by selecting a cancel control 288. Theuser can alternatively choose to save the note 287 by selecting a savecontrol 289. The user can add or alter notes from other cells byselecting one from the scroll area 282.

Referring now to FIG. 2I, an overlay 290 is depicted. The overlay 290can be presented if, for example, a user selects on the cell 208 (FIG.2E) when in a flipped state (e.g., the user selected flip control 262(FIG. 2D)). As with overlay 274, overlay 290, or similar overlay, maysimilarly be provided when the icon 208 in FIG. 2C (relevant to allstudents) is selected. Such an overlay may provide the same, or similar,functionality with respect to the group of students as overlay 290.

Here, the flipped cell 208 is shown as an interactive graphic 291. Theoverlay 290 includes an interactive detail area 292. The detail area 292includes the text of the objective description (e.g., “6.N.16—Add andsubtract fractions with unlike denominators”). Other descriptive detailscan be included in reference to the associated learning objectives. Forexample, a period notation associated with the period may be included,as well as the resource links, as described above.

The detail area 292 includes a statistics section 293. The statisticssection 293 includes graphics that illustrate the proportion of studentsat each student-oriented performance level on a class, a school, or aregion basis. In the example shown, the statistics section 293 providespie chart graphs, like pie chart 257, to depict a proportion of students(i) at risk (ii) at mastery, or (iii) at non-mastery at each of theclass, school, and region group levels. The pie chart 257 is exemplaryand as such, different statistics can also be detailed using a similargraphic or additional graphics. In some implementations, the pie chartscan show data in each period such as IA1, IA2, and IA3.

At some point, a user can make a selection on the interactive graphic291. As shown, the user has selected item (#16) 294. In response toselecting item 294, the question (i.e., assessment item) correspondingto item 294 is displayed in a control 295. Referring now to FIG. 2J, theuser selected item (#17) 296 and the control 295 updated accordingly. Ingeneral, the user can select each item identified by number (#16-19) inthe cell on the left and the text of that item (e.g., questions) areshown at the right in the control 295. Alternatively, the user can usecontrol 295 to scroll through each item or question listed on thegraphic 291. Similar to the previous overlay 274, the overlay 290 alsoincludes a scroll area 297 which can be used to select other cells.

Referring now to FIG. 2K, an overlay 298 is depicted. This examplepertains to a “constructed response” (i.e., an open response) item. The“constructed response” item may mean that a student has been asked toproduce a written response to a question or prompt. The overlay 298pertains to a different standard (6.G.3) for the student “JasonBarnett.” Here, the user has selected an option on overlay 290 to viewthe student's actual work product. This may be a scanned image of thestudent's pencil-and-paper based administration, or typed text forcomputer-based administration, for example. This may also include audioor video of a student performing a task. While not shown in FIGS. 2I and2J, the overlay 290 can include a link to the student's work productassociated with control 295. Selecting the link presents the student'swork as another overlay 299. The overlay 299 can be analyzed by ateacher or other entity. This can provide the advantage of having eachtest question and answer or work product on hand when reviewing studentresults. Thus, the user may not be required to find a copy of the test.

In some implementations, a student's “selected constructed response” maybe provided. The “selected constructed response” may represent amultiple choice item. With the multiple choice item, a list of answerscan be provided including a correct answer and several incorrect answers(e.g., distractors). In some implementations, a list of students can beprovided detailing which student made each choice and a description of aparticular misconception that may have led a particular student tochoose an incorrect answer.

Also, while the examples show a display of the student's response(answer to the assessment item) and the student's work product, otherimplementations may display the student's responses to one or more ofthe assessment items with or without showing the work product. Forinstance, for multiple choice assessment, the display may show only theitem selected by the student, without showing the work the studentperformed to arrive at the selection (although other implementations mayshow both).

The plurality of screenshots described above can be adjusted to providevarious mechanisms for collecting and displaying data regarding a groupor individual. The information collected and/or displayed can beindividual-oriented when an individual is selected and group-orientedwhen a group is selected. For example, while some of the informationdisplayed in overlay 274 is depicted as being group-oriented even thoughan individual is selected, in other implementations the information (forexample, the plan and notes) may be individual-oriented. The informationcollected and/or displayed can also be a combination of bothindividual-oriented and group-oriented information regardless of whetheran individual or group is selected. Moreover, any combination of groupand individual data can be depicted in the same application or screenregardless of the actual application or screen invoked.

FIG. 3A illustrates a screenshot showing an example of a user interface300 displaying an interactive learning map. The user interface 300includes a learning map graphic 302 and a palette 304. Similar to thelearning map graphic 202 shown in FIG. 2A, each cell of the learning mapgraphic 302 can be color-coded or pattern-coded depending on performanceresults attained by a student or a group of students on assessment itemsfor a corresponding learning objective. As shown in FIG. 3A, thelearning map graphic 302 corresponds to a particular student. Forexample, a particular student can be selected from a list of students(not shown). As another example and as described in more detail below, astudent can be selected using a histogram 305. The color-coding of thelearning map graphic 302 may indicate a level of performance of thestudent on one or more assessment items for a corresponding learningobjective. For example, cells 306 and 308 are each colored green toillustrate the student has reached the mastery target of a respectivelearning objective.

The user interface 300 may also be used to display performance resultsfor a class or other group of students. For example, a mastery level canbe determined for each cell, for each student, and a class performancecan be determined for each cell, such as based on the percentage ofstudents at a particular mastery level. For example, if 85% of thestudents are at a mastery target for a particular cell, it can bedetermined that the class has also reached the mastery target.Accordingly, the cell can be colored green or given some otherindication of mastery level achievement, in the learning map graphic302.

For some assessments, a single assessment may not provide data for eachcell of the learning map. For example, a single assessment may providedata for some, but not all cells. When evaluating student performance,multiple sources of different types of assessment data can be used. Suchan approach can result in population of more cells of the learning mapwith performance information than if data from a single assessment isused. As mentioned above, assessment types can include interimassessments, state assessments tests, national assessment tests,homework assignments, classwork assignments, class quizzes, diagnosticassessments, formative assessments, summative assessments, end ofchapter tests or quizzes, exit ticket assessments, or other types ofassessments.

Assessments used to populate a learning map can be selected using atimeline control 309 included in the palette 304. The timeline control309 displays date indicators and also displays multiple assessmentindicators 310, where each assessment indicator 310 is displayed at aposition on the timeline 309 corresponding to the date that anassociated assessment was conducted. For example, an assessmentindicator 310 a indicates that a homework assessment occurred on 8/31,an assessment indicator 310 b indicates that an ad-hoc assessmentoccurred on 9/10, and an assessment indicator 310 c indicates that abenchmark test assessment occurred on 10/2.

A user can select an assessment indicator 310 to populate the learningmap with performance information for the associated assessment. The usercan select multiple assessment indicators 310 to populate the learningmap with performance information for multiple assessments. To selectmultiple assessment indicators 310, the user can, for example, press a“control” key while selecting each of multiple assessment indicators310.

The histogram 305 indicates a distribution of performance results forthe selected assessment(s), for the class that includes the selectedstudent. Each rectangle included in the histogram 305 represents astudent in the class. A selected rectangle 312 represents the selectedstudent. The horizontal placement of the rectangle 312 indicates theperformance of the student on the selected assessment(s), and the colorof the rectangle 312 represents a mastery level of the student'sperformance (e.g., met mastery target, approaching mastery target,failed mastery target). A different rectangle can be selected to viewdetails corresponding to performance of another student.

In some implementations, to select assessments, the user can select andposition a slider control 313 on the timeline control 309. For example,the slider control 313 is currently positioned on the timeline control309 at a position corresponding to a date of 8/20. In this example, allavailable assessments with a date after the date associated with theslider control 313 can be used to populate the learning map withperformance information. For example, the assessments associated withassessment indicators 310 a, 310 b, and 310 c can be automaticallyselected since they are to the right of the slider control 313. The usercan move the slider control 313 to a different date to selectassessments occurring after that date.

For example and referring to FIG. 3B, a user can move the slider control313 on the timeline control 309 so that the slider control 313 is afterthe assessment indicators 310 a and 310 b and before the assessmentindicator 310 c. In this example, the assessment corresponding to theassessment indicator 310 c, but not the assessments corresponding to theassessment indicators 310 a and 310 b, is used to populate the learningmap.

Performance results for a given cell may change as the selection ofassessments changes, and performance indicators (e.g., coloring) of thelearning map may be correspondingly updated. For example, while the cell306 shown in FIG. 3A indicates mastery of the associated learningobjective based on the assessments corresponding to selected assessmentindicators 310 a, 310 b, and 310 c, the red color of the correspondingcell 306 shown in FIG. 3B indicates that the student failed mastery ofthe learning objective based on performance of the benchmark testassessment associated with the selected assessment indicator 310 c.

The histogram 305 shown in FIG. 3B shows information about theperformance on the benchmark test assessment corresponding to theassessment indicator 310 c, for the class including the student. Thehistogram 305 shown in FIG. 3B displays a different distribution thanthe histogram 305 shown in FIG. 3A, since the histogram 305 shown inFIG. 3A displays a distribution corresponding to class performance on adifferent set of assessments (e.g., assessments corresponding toassessment indicators 310 a, 310 b, and 310 c).

In some implementations, a set of controls 314 can be used to configurewhich types of assessments are used to populate the learning map withperformance information. A user can use the controls 314 to select ordeselect one or more assessment types. For instance, the example of thecontrols 314 shown in FIG. 3B indicate that classwork assignments andclass quizzes are not selected. In implementations where both thetimeline control 309 and the controls 314 are used, an assessment may beselected for populating the learning map if the assessment occurs on adate after the date represented by the slider control 313 and if thecontrols 314 indicate that the type of the assessment is selected.

For each question on each of the selected assessments, the learningobjective(s) to which the question relates can be determined.Performance information for a particular cell can be determined byanalyzing performance information for one or more related questionsincluded on one or more assessments. In some implementations, eachquestion is weighted equally, regardless of the type of question andregardless of which assessment includes the question. In someimplementations, some questions and some assessments are weighteddifferently. Weighting is described in more detail below with respect toFIG. 5.

As mentioned, multiple, different types of assessments may be used topopulate performance information for a particular cell of the learningmap. A cell may be selected to view details related to a particularassessment or assessments. For example, FIG. 4A illustrates an examplescreenshot showing an overlay 402 related to cell 306. The overlay 402can be navigated to, for example, by selecting the cell 306 shown inFIG. 3A, for instance, by clicking a mouse over the icon 306. In theparticular example shown, the cell 306 and related overlay 402correspond to a learning objective with identifier “6.N.16” relating toadding and subtracting fractions with unlike denominators. The overlay402 can show information relating to a particular student with respectto this learning objective. The overlay 402, or similar overlay, mayprovide the same, or similar, functionality with respect to a group ofstudents.

The overlay 402 includes a graphic 404 of the cell 306. The green colorof the graphic 404 indicates that the student achieved mastery of theassociated learning objective based on performance results of theselected assessments. An assessment selection control 406 listsassessments that are used to determine performance for the learningobjective. For example, the assessment selection control 406 indicatesthat performance determination is based on an ad-hoc assignment, abenchmark test, and a homework assignment. That is, each of theassessments associated with the assessment indicators 310 a, 310 b, and310 c shown in FIG. 3A relate to the learning objective associated withthe cell 306. As discussed below, for some learning objectives, someavailable assessments may not relate to the learning objective and aretherefore not listed in the assessment selection control 406.

The user can select an assessment from the assessment selection control406 to view information about the selected assessment. For example, anassessment information area 408 shows, for a selected benchmark testassessment, charts showing a performance distribution for the selectedclass, the school including the class, and the region associated withthe school. Other assessment information may be shown in or may beaccessible from the assessment information area 408. For example,assessment questions and answers may be shown, such as in a mannersimilar to FIGS. 2E and 2I-2K as described above. In further detail, inplace of or in addition to distribution charts, questions on theselected assessment that relate to the learning objective of the cellcan be displayed, along with the student's answers to the questions andcorrect or expected answers to the questions.

As another example, FIG. 4B illustrates an example screenshot showingthe overlay 402 in association with cell 308. The overlay 402 can benavigated to, for example, by selecting the cell 308 shown in FIG. 3A.The overlay 402 shown in FIG. 4B illustrates details about a learningobjective with identifier “6.N.17b” relating to dividing fractions withunlike denominators. The green color of the graphic 404 indicates thatthe student achieved mastery of the associated learning objective basedon performance results of the selected assessments.

The assessment selection control 406 lists assessments that are used todetermine performance for the learning objective. For example, theassessment selection control 406 indicates that performancedetermination is based on an ad-hoc assignment and a benchmark test.Notably, a homework assessment associated with the assessment indicator310 a shown in FIG. 3A is not displayed in the assessment selectioncontrol 406, indicating that the homework assessment does not relate tothe learning objective “6.N.17b”. In general, the number and type ofassessments used for a particular cell may differ from the number andtype of assessments used for a different cell.

FIG. 5 is a flow chart showing an example of a process 500 fordetermining mastery of a learning objective based on multipleassessments. The process 500 can be performed by, for example, theserver system 102, and for clarity of presentation, the description thatfollows uses the system 100 as the basis for describing the process 500.However, another system, or combination of systems, may be used toperform the process 500.

The server system 102 selects a learning objective (502). The serversystem 102 may, for example, select a learning objective in response touser input. For example, a teacher may, using an interface displayed onthe teacher client system 104, select a particular cell of a learningmap, where the cell is associated with a particular learning objective.As another example, a teacher may, using an interface displayed on theteacher client system 104, initiate a change of one or more parameters(e.g., assessments, student, class or other student grouping) affectingdisplay of performance information in a learning map, resulting in thepopulation of performance information for each cell of the learning map,to correspond to the change of parameters. Each cell may be selected inturn. That is, a particular cell may be selected, and the process 500may be performed for the selected cell, and subsequently for each cellof the learning map.

The server system 102 may select a learning objective other than inresponse to user input. For example, the server system 102 may select alearning objective on a periodic basis (e.g., every minute, hourly,daily) as part of an automatic, periodic process used to update alearning map. As another example, the server system 102 may select alearning objective in response to receiving new assessment information.

The server system 102 accesses assessment information (504). Forexample, the server system 102 may access assessment information 112a-112 d, such as from the repository 118. Each assessment information112 a-112 d can include information for a different type of assessment.The server system 102 may identify which assessments pertain to theselected learning objective. Additionally, for each assessment whichpertains to the selected learning objective, the server system 102 mayidentify which question(s) of the assessment pertain to the selectedlearning objective. The server system 102 may consult a mapping table toidentify which assessments and/or which questions pertain to theselected learning objective.

The server system 102 may determine whether performance data is to bedetermined for a student or for a group of students. If performance datais to be determined for a particular student, the server system 102 mayaccess assessment data for the student, for the assessments and/orquestions which pertain to the selected learning objective. Ifperformance data is to be determined for a group of students, the serversystem 102 may access assessment data for each student, for theassessments and/or questions which pertain to the learning objective.

The server system weights assessment scores for the assessment items onthe assessments related to the selected learning objective (506). Ingeneral, assessment scores may be weighted based on how well anassessment item indicates a student's mastery of the selected learningobjective. For example, a weight assigned to assessment items in a highstakes test may be higher than a weight assigned to assessment items ina homework assessment. As another example, the server system 102 mayweight newer assessments higher than older assessments. For example,assignments that are less than two weeks old may be weighted at 100%,assignments between two and four weeks old may be weighted at 80%,assignments between four and six weeks old may be weighted at 60%, andassignments older than six weeks may be weighted at 0%. Assessment itemsthat are considered to be a better differentiator of a student'sperformance on the selected learning objective relative to otherassessment items may be weighted more than those other assessment items.

Assessments and/or individual assessment items may be weighted based ondifficulty. For example, a mapping can be used that identifies a scorefor an assessment or question which indicates mastery. For example, forsome assessments or questions, mastery may be identified as a score of90%, whereas for other, more difficult assessments, mastery may beidentified as a score of 70%. Rasch modeling can be used for weightingscores based on difficulty.

The server system 102 determines overall performance score(s) (508). Forexample, the server system 102 may, for each student, determine a totalweighted score for the selected learning objective based on the weightedscores for the assessment items related to the selected learningobjective, and determine a total weighted possible score for those sameassessment items. The server system 102 may, for each student, determinean overall performance score for the student by calculating a ratio ofthe total weighted score to the total weighted possible score. As anexample, suppose there are two assessment items related to the learningobjective, with one having a total possible points of 5 and the otherhaving a total possible points of 1, and that the weight applied to thefirst assessment item is 100% (or 1) and the weight applied to thesecond assessment item is 80% (or 0.8). Suppose also that the studentachieved 3 points on the first assessment item and 1 point on the secondassessment item. The server system 102 may then determine the overallperformance score for the selected learning objective as(1*3+0.8*1)/(1*5+0.8*1), which is equal to 0.66.

The server system 102 determines a mastery level for the learningobjective, for example, based on the overall performance score (510). Ifperformance data is to be determined for a single student, the serversystem 102 may compare the overall performance score for the student toone or more threshold scores, where a threshold score indicates aparticular mastery level. For example, a threshold score of 85% mayindicate a mastery level, meaning scores at or above the threshold of85% indicate mastery of the learning objective. A threshold score of 70%may indicate approaching a mastery level, meaning scores at or above 70%but less than 85% indicate approaching master level and also meaningscores below 70% indicate failure to obtain mastery of the learningobjective.

If performance data is to be determined for a group of students, amastery level for the group may be determined. As described above,calculating mastery level for a group may be defined in terms of thepercentage of students who met a threshold or may be based on an averagepoints per student in the group.

FIG. 6 shows an example of a generic computer device 600 and a genericmobile computer device 650, which may be used with the techniquesdescribed here. Computing device 600 is intended to represent variousforms of digital computers, such as laptops, desktops, workstations,personal digital assistants, tablets, servers, blade servers,mainframes, and other appropriate computers. Computing device 650 isintended to represent various forms of mobile devices, such as personaldigital assistants, cellular telephones, smartphones, and other similarcomputing devices. The components shown here, their connections andrelationships, and their functions, are meant to be exemplary only, andare not meant to limit implementations of the inventions describedand/or claimed in this document.

Computing device 600 includes a processor 602, memory 604, a storagedevice 606, a high-speed interface 608 connecting to memory 604 andhigh-speed expansion ports 610, and a low speed interface 612 connectingto low speed bus 614 and storage device 606. Each of the components 602,604, 606, 608, 610, and 612, are interconnected using various buses, andmay be mounted on a common motherboard or in other manners asappropriate. The processor 602 may process instructions for executionwithin the computing device 600, including instructions stored in thememory 604 or on the storage device 606 to display graphical informationfor a GUI on an external input/output device, such as display 616coupled to high speed interface 608. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices600 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 604 stores information within the computing device 600. Inone implementation, the memory 604 is a volatile memory unit or units.In another implementation, the memory 604 is a non-volatile memory unitor units. The memory 604 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 606 is capable of providing mass storage for thecomputing device 600. In one implementation, the storage device 606 maybe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product may be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer-readable ormachine-readable medium, such as the memory 604, the storage device 606,memory on processor 602, or a propagated signal.

The high speed controller 608 manages bandwidth-intensive operations forthe computing device 600, while the low speed controller 612 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 608 iscoupled to memory 604, display 616 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 610, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 612 is coupled to storage device 606 and low-speed expansionport 614. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 600 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 620, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 624. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 622. Alternatively, components from computing device 600 may becombined with other components in a mobile device (not shown), such asdevice 650. Each of such devices may contain one or more of computingdevice 600, 650, and an entire system may be made up of multiplecomputing devices 600, 650 communicating with each other.

Computing device 650 includes a processor 652, memory 664, aninput/output device such as a display 654, a communication interface666, and a transceiver 668, among other components. The device 650 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 650, 652,664, 654, 666, and 668, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 652 may execute instructions within the computing device650, including instructions stored in the memory 664. The processor maybe implemented as a chipset of chips that include separate and multipleanalog and digital processors. The processor may provide, for example,for coordination of the other components of the device 650, such ascontrol of user interfaces, applications run by device 650, and wirelesscommunication by device 650.

Processor 652 may communicate with a user through control interface 658and display interface 656 coupled to a display 654. The display 654 maybe, for example, a TFT LCD (Thin-Film-Transistor Liquid CrystalDisplay), an OLED (Organic Light Emitting Diode) display, or anelectronic ink display, or other appropriate display technology. Thedisplay interface 656 may comprise appropriate circuitry for driving thedisplay 654 to present graphical and other information to a user. Thecontrol interface 658 may receive commands from a user and convert themfor submission to the processor 652. In addition, an external interface662 may be provided in communication with processor 652, so as to enablenear area communication of device 650 with other devices. Externalinterface 662 may provide, for example, for wired communication in someimplementations, or for wireless communication in other implementations,and multiple interfaces may also be used.

The memory 664 stores information within the computing device 650. Thememory 664 may be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 674 may also be provided andconnected to device 650 through expansion interface 672, which mayinclude, for instance, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 674 may provide extra storage space fordevice 650, or may also store applications or other information fordevice 650. Specifically, expansion memory 674 may include instructionsto carry out or supplement the processes described above, and mayinclude secure information also. Thus, for example, expansion memory 674may be provide as a security module for device 650, and may beprogrammed with instructions that permit secure use of device 650. Inaddition, secure applications may be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 664, expansionmemory 674, memory on processor 652, or a propagated signal that may bereceived, for example, over transceiver 668 or external interface 662.

Device 650 may communicate wirelessly through communication interface666, which may include digital signal processing circuitry wherenecessary. Communication interface 666 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 668. In addition, short-range communication may occur, suchas using a Bluetooth, Wi-Fi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 670 mayprovide additional navigation- and location-related wireless data todevice 650, which may be used as appropriate by applications running ondevice 650.

Device 650 may also communicate audibly using audio codec 660, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 660 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 650. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.), and may also includesound generated by applications operating on device 650.

The computing device 650 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 680. It may also be implemented as part of asmartphone 682, personal digital assistant, a laptop, or other similarmobile device. In some implementations, the computing device 650 may beimplemented in a non-mobile computing device such as a desktop, multipleservers, or other similar non-mobile device.

Device 650 may also include one or more different devices that arecapable of sensing motion. Examples include, but are not limited to,accelerometers and compasses. Accelerometers and compasses, or otherdevices that are capable of detecting motion or position are availablefrom any number of vendors and may sense motion in a variety of ways.For example, accelerometers may detect changes in acceleration whilecompasses may detect changes in orientation respective to the magneticNorth or South Pole. These changes in motion may be detected by thedevice 650 and used to update the display of the respective devices 650according to processes and techniques described herein.

Various implementations of the systems and techniques described here maybe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations may include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and may be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here may be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user may provideinput to the computer. Other kinds of devices may be used to provide forinteraction with a user as well; for example, feedback provided to theuser may be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user may bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here may be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usermay interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system may be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system may include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made. In addition, thelogic flows depicted in the figures do not require the particular ordershown, or sequential order, to achieve desirable results. In addition,other steps may be provided, or steps may be eliminated, from thedescribed flows, and other components may be added to, or removed from,the described systems. Accordingly, other embodiments are within thescope of the following claims.

What is claimed is:
 1. A computing system comprising: one or moreprocessing devices; and one or more storage devices storing instructionsthat, when executed by one or more processing devices, cause the one ormore processing devices to: display one or more learning map interfaces,the learning map interfaces including a selection control configured toallow a user to select a time frame during which a subset of assessmentsfrom a set of assessments were administered to a student, the subsetincluding less assessments than the set; receive a selection from theuser using the selection control, the selection indicating the timeframe; access assessment information for the student, the assessmentinformation including information related to assessments in the subset;populate a learning map with the performance information for thestudent, the learning map including a plurality of cells, each cellcorresponding to a learning objective and the plurality of cells beingarranged in a manner that reflects a relationship between the learningobjectives corresponding to the cells, wherein, to populate the learningmap with performance information, the instructions cause the one or moreprocessing devices to; determine, for each assessment in the subset andfor each assessment item on a given assessment in the subset, a learningobjective that is associated with the assessment item, includingdetermining a first set of assessment items associated with a firstlearning objective and a second set of assessment items associated witha second learning objective, wherein the first set of assessment itemsincludes different assessment items than the second set of assessmentitems; determine a first performance level, from among multipleperformance levels, of the student based on the first set of assessmentitems; modify a first one of the cells to include a first indication ofthe first performance level, the first cell corresponding to the firstlearning objective; determine a second performance level, from among themultiple performance levels, of the student based on the second set ofassessment items; and modify a second one of the cells to include asecond indication of the second performance level, the second cellcorresponding to the second learning objective; and display one or morelearning map interfaces, the learning map interfaces being configured todisplay the learning map populated with the performance information. 2.The system of claim 1 wherein the selection control comprises a timelinecontrol that allows a user to select a time such that assessments in theset of assessments administered to the student after the selected timeare included in the subset of assessments and assessments in the set ofassessments before the selected time are not included in the subset. 3.The system of claim 1 wherein the subset of assessments includesmultiple, different types of assessments.
 4. The system of claim 1wherein, to determine the first performance level, the instructionsinclude instructions that cause the one or more processing devices to:access a set of assessment scores corresponding to the first set ofassessment items; apply weights to the accessed assessment scores togenerate weighted assessment scores; determine the first performancelevel based on the weighted assessment scores.
 5. The system of claim 4wherein the weights vary based on a time at which the assessment itemswere administered to the student.
 6. The system of claim 4 wherein theweights vary based on a difficulty of the assessments items.
 7. Thesystem of claim 4 wherein the weights vary based on an importance of theassessment on which the assessment items were administered.
 8. Thesystem of claim 1 wherein the weights vary based on how well theassessment items differentiate mastery levels.
 9. A method comprising:displaying, on a display device, one or more learning map interfaces,the learning map interfaces including a selection control configured toallow a user to select a time frame during which a subset of assessmentsfrom a set of assessments were administered to a student, the subsetincluding less assessments than the set; receiving, using one or moreprocessing devices, a selection from the user using the selectioncontrol, the selection indicating the time frame; accessing, using theprocessing device, assessment information for the student, theassessment information including information related to assessments inthe subset; populating, using the processing devices, a learning mapwith the performance information for the student, the learning mapincluding a plurality of cells, each cell corresponding to a learningobjective and the plurality of cells being arranged in a manner thatreflects a relationship between the learning objectives corresponding tothe cells, wherein populating the learning map with performanceinformation includes: determining, using the processing devices and foreach assessment in the subset and for each assessment item on a givenassessment in the subset, a learning objective that is associated withthe assessment item, including determining a first set of assessmentitems associated with a first learning objective and a second set ofassessment items associated with a second learning objective, whereinthe first set of assessment items includes different assessment itemsthan the second set of assessment items; determining, using theprocessing devices, a first performance level, from among multipleperformance levels, of the student based on the first set of assessmentitems; modifying, using the processing devices, a first one of the cellsto include a first indication of the first performance level, the firstcell corresponding to the first learning objective; determining, usingthe processing devices, a second performance level, from among themultiple performance levels, of the student based on the second set ofassessment items; and modifying, using the processing devices, a secondone of the cells to include a second indication of the secondperformance level, the second cell corresponding to the second learningobjective; and displaying, on the display device, one or more learningmap interfaces, the learning map interfaces being configured to displaythe learning map populated with the performance information.
 10. Themethod of claim 9 wherein the selection control comprises a timelinecontrol that allows a user to select a time such that assessments in theset of assessments administered to the student after the selected timeare included in the subset of assessments and assessments in the set ofassessments before the selected time are not included in the subset. 11.The method of claim 9 wherein the subset of assessments includesmultiple, different types of assessments.
 12. The method of claim 9wherein determining the first performance level includes: accessing,using the processing devices a set of assessment scores corresponding tothe first set of assessment items; applying weights, using theprocessing devices, to the accessed assessment scores to generateweighted assessment scores; determining, using the processing devices,the performance level based on the weighted assessment scores.
 13. Themethod of claim 12 wherein the weights vary based on a time at which theassessment items were administered to the student.
 14. The method ofclaim 12 wherein the weights vary based on a difficulty of theassessments items.
 15. The method of claim 12 wherein the weights varybased on an importance of the assessment on which the assessment itemswere administered.
 16. The method of claim 9 wherein the weights varybased on how well the assessment items differentiate mastery levels.